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Abstract — wheelchair is a mobility device designed for 
shifting patients, moving physically challenged people from one 
place to another. This paper includes the electronic wheelchair 
that implemented for disabled person to eliminate the attendee 
required to propel wheelchair. Controlling of wheelchair is fully 
depends on eye movement and emergency switch. A camera 
situated on wheelchair in front of the user for capturing the 
image of eye and tracks the eye pupil position by image 
processing techniques. According to the eye movement of user, 
motor will be move in respective directions such as left, right 
and forward. Ultrasonic sensor is mounted in front of 
wheelchair for safety purpose to detect static and dynamic 
barriers with some specific range of distance. Any obstacle in 
this specific range, wheelchair movement stops automatically. A 
switch is mounted on wheelchair for emergency purpose to stop 
in required direction if someone wants to interrupt or require 
attention on them. This is independent and low cost wheelchair 
system. A raspberry pi board is brain of the system. 

Index Terms — Image processing, Mobility device, Open 
computer vision image library, Python, Raspberry pi, Switch, 
Wheelchair. 


i. Introduction 

There are various types of wheelchair like manual wheelchair, 
electric wheelchair. Manual wheelchairs are driven with help 
of man power as source of energy for moving the chair, these 
are self propelled or propelled with the help of attendee. The 
wheelchair that runs by means of electric motor is known as 
electric powered wheelchair. This wheelchair requires 
navigational controls, usually a small joystick mounted on the 
armrest for users who can not manage it manual joystick, head 
switches are provided and chin operated joysticks are 
provided , other specialist controls are also provided for 
independent operation of the wheelchair [21]. 

Eye movement controlled wheelchair is to enable completely 
paralyzed patient as well as elderly to make their life more 
accessible [1]. Person who are unable to walk and are using 
wheelchair exert great amount of energy using physical 
strength to turn the wheels. Disabled would save energy and 
could use their hand and arm for other activities. Currently 
there are different eye based method will be use for 
controlling wheelchair such as EOG, ECG, EEG based, 
Eyeball sensing method [4] -[5] -[17]. To decide the location 
of eye pupil depends on voltage variation. But for different 
output voltage will be generates for different user, which 
gives wrong location of the eye pupil [6] .Voice activated 
power wheelchair system, which works properly when user 
speak the command clearly, according to it left, right, 
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forward, back and stop. Other voices which come from 
surrounding user may affect the system [19]. The head 
movement based system and chin control based system, bad 
movement gives problem [7]-[8]-[17]. Sip and Puff 
wheelchair system, not good for people with weak breathing. 
Infrared reflection based eye pupil detection system provide 
accurate detection of eye pupil centre location. But the 
infrared radiation affects the eye and people may loss the eye 
visibility [20]. Eye controlled wheelchair system is 
introduced using camera for capture the image [16]. But 
someone required attention on itself, so here we use central 
switch to stop. 

Camera captured the image in real time based on face, eye and 
eye pupil detection with minimum delay of time and analyzes 
the image as input to set the commands to interface the motor 
driver IC through sending the command to GPIO pins, to 
perform the different operation such as left, right, forward and 
stop. 


Image processing open computer vision (open CV) library is 
used for face and eye detection [2]. System includes 
multistage that is track the eye pupil centre [9]. To detect the 
single or multiple face and detection of both Eyes, this is 
ultimate goal of this system. Several Algorithms are used to 
find exact pupil location direction. Haar cascade like feature 
detection algorithm is used [3]. Image processing are includes 
face detection, eye detection, color image to gray conversion, 
blurring, edge detection, pattern matching , filtering, noise 
reduction, etc. Figure 1 shows the architecture diagram of the 
system. 
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Figure 1: System Architecture Diagram 


Raspberry pi board is soul of the system, which control the 
complete system operation. Image processing based data 
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signal sent to the raspberry pi, raspberry pi received the data 
and analyze it and send the control signal to motor driving 
circuit, based on the location of eye pupil. This will decide 
motor run either in clockwise or anticlockwise direction or 
stop. Two individual motors are fixed on each wheel. 
Ultrasonic sensor is mounted on the wheelchair for obstacle 
detection. If sensor gets any obstacle very close to the 
wheelchair, it indicates to the raspberry pi and it will sends the 
signal to motor driving circuit to stop the motor. 

II. HARDWARE DESIGN MODEL 

In this system there is mandatory to gives the power supply to 
individual components and standard power supply should be 
used for raspberry pi, camera, sensor, motors and switch. The 
figure 2 shows the functionality of the system. 



Figure 2: Design Model 

In proposed system component like LCD, camera, power 
circuit and Wi-Fi adapter directly connected through the 
raspberry pi board and raspberry pi board is connected to the 
web server for remote access [10], in case of emergency 
controlling or check the status of wheelchair. 

1) HARDWARE DESCRIPTION 

A) Raspberry pi board 

Raspberry pi board is brain of the system. Raspberry pi board 
have its own operating system is known as raspbian which is 
Linux based operating system and compatible with raspberry 
pi board. A real time data receive and determine the digital 
data by raspberry pi B+ model board, which is very efficiently 
work with the multiple images. Raspberry pi sends the 
command to motor driver which is enabling the GPIO pin to 
raspberry pi. 


B) Web camera 

Web camera is used for capturing the image. We can also use 
HD (high definition) camera but it increase the memory size, 
system can’t able to read the image and it will increase the 
processing time. UV4L driver is needed for interfacing a 
camera with raspberry pi board. 

C) Ultrasonic sensor 

Ultrasonic sensor is used to detect obstacle in the path of 
wheelchair. Sensor is directly connected to the raspberry pi 
board. It receives the data and measuring the distance between 
wheelchair and obstacle. If any obstacle is detected very close 
to wheelchair, motors will stop to run the wheels. Ultrasonic 
sensor is a very affordable proximity / distance sensor that has 
been used mainly for object avoidance in various robotics 
projects. 

D) Motor 

Two 12 v DC motor is used in project to demonstrate running 
of wheelchair in forward, reverse, left and right direction. 
L293D motor driver is used to interface with raspberry pi 
which is TTL compatible. Two H bridges of L293D can be 
connected in parallel to increase its current capacity to 2 Amp. 

E) LCD (Liquid crystal display) 

LCD is very helpful in providing user interface as well as for 
debugging purpose. Most common LCD controller is 
HITACHI 44780 which provides a simple interface between 
the controller and LCD. LCD is used as a monitor in most of 
the electronic project. 

F) Relay 

Relay is an electromechanical device and basically consists of 
an electromagnet and a number of contact sets. Relay is used 
in this system to change the direction of motor very fast 
without using the finger. 

2) SOFTWARE DESCRIPTION 

A) Putty software 

Putty is a free and open source terminal emulator and network 
file transfer application. Putty software is used to connect the 
desktop to raspberry pi board. 

B) OpenCV image library 

OpenCV is released under a BSD license and free for both 
academic and commercial use. It has C++, C, Python and Java 
interfaces and supports Windows, Linux, Mac OS, iOS and 
Android. OpenCV was designed for computational efficiency 
and with a strong focus on real time application. 

C) Python language 

Python is an interpreter, object oriented, high level 
programming language with dynamics semantics. Its high 
level built in data structures, combined with dynamic typing 
and dynamic binding; make it very attractive for rapid 
application development. Python’s easy to learn syntax 
emphasizes readability and therefore reduce the cost of 
program maintenance. The fast edit - test -debug cycle makes 
this very effective. Moreover Matlab is used for coding but it 
is quite expensive and algorithms are proprietary, math work 
puts restriction on code portability. 
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3) DESIGN METHODS 

Algorithm used for detecting the eye pupil location by image 
processing. Several stages used to determine eye movement 
such as face and eye detection, colour to gray conversion, 
canny edge detection, Hough circle transform and eye 
tracking. Initially the system received the captured images by 
USB web camera. Our first aims to detect the user face 
accurately. If there are multiple faces, it will show the error. 
System indicates the face of user in a specific area of image. 
After that it will perform the several operation of image 
processing to track the eye movement. Figure 3 shows the 
proposed methodology of implemented system. 



Figure 3: Proposed method 


Camera will start to capture the images. For the face and eye 
detection Haar cascade algorithm is used. After detection of 
face, it will try to detect the eye inside the face and draw the 
rectangular box over the eye. To detect the eye pupil and 
define its centre points is ultimate goal of the system by 
several images processing technique. The system will crop the 
eye region of interest and it will draw the all possible circle on 
that particular area to detect the eyeball. Then we applied 
corner detection method to detect the comer. Average of these 
two points indicates the centre point. To measure the distance 
between the centre point and eye circle centre point using 
coordinates system. Minimum distance indicates the eye pupil 
in left and maximum value indicates the eye pupil presented in 
right. If there is no movement of eye it will indicates eye is in 
middle position [11]. When eye move in left, left side motor 
will run and when eye move in right, right side motor will run. 
If eye will be in centre, both motors moved and wheelchair 
moving in forward direction. If any obstacle is detected, 
system will be stop and move either in left or right direction 
according to eye movement. If someone calls the user, user 
will stop the wheelchair by emergency switch. Eye blinking 
logic will be decide the start and stop operation [12]. For edge 
detection canny edge detection is used. For circle detection 
Hough circle transform method is used. Image processing 
based on openCV library is installed in raspberry pi memory. 
Figure 4 shows the flow chart of system. 



III. IMPFEMENT ATION AND SYSTEM DESCRIPTION 


The low power consumption credit card sized raspberry pi B+ 
board is used, which have inbuilt 40 pin GPIO, 4 USB ports, 
UART, PWM, HDMI port and Ethernet adapter port for 
internet connection via wireless or wired connection. 
Raspberry pi have a 512 MB RAM and capable of up to 32 
GB external memory, controlled based on ARM architecture. 
Camera is directly connected with raspberry pi board and 
continuously captures the images, distance between eye and 
camera device is fixed. It may be 10 to 15 cm. Face and eye 
detected by Haar cascade algorithm and find out the exact 
pupil location. Then several algorithms used to measure the 
centre point from the average of both corners of the eye. This 
gives the correct information eye movements. The motor 
driving circuit is connected with raspberry, battery for power 
supply of motors, relay for controlling the motor driving IC. 
System continuously generates the directive signal to enable 
the GPIO pins and perform the required operation like left, 
right, forward and stop. Central switch is also connected with 
system for emergency purpose. Ultrasonic Sensor for obstacle 
detection. 

IV. SYSTEM INSTALLATION 

To boot a train image in micro SD card Win32 diskimager 
software used. Then putting a bootable memory device on 
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raspberry pi board and admit the raspbian operating system 
directly without resetting [16]. 

V. EYE TRACKING 

There is several image processing operation performed in 
system, such as 

1 ) BGR to Gray Conversion 

A colored image is converted in to gray image to reduce the 
system delay time because image frame size should be low for 
proper processing. 



a) Gray images 

2 ) Feature Detection and Blurring Image 
The Gaussian blur filter is used for blurring the image, which 
helps to detect the exact edges of specific area of the cropped 
image. 



b) Gaussian blurred filter image c) feature detection on 
image 


3) Canny Edge Detection 

Canny edge detection algorithm used a blurred image. It will 
be applied for soft edges of the image. 



d) Canny edge detection 

4) Hough Circle Transform 

We use Hough circle transform method to draw a circle on eye 
pupil. 



5) Eye Tracking 

To track the eye movement we use coordinate system. Which 
decide the eye centre point location [13]-[14]-[16]. Figure 6 
indicates the eye pupil location using coordinate system 
graph. 

Where A1 and A2 is corner point of eye pupil in X direction, 
B1 and B2 is corner point of eye pupil in Y direction. The X 
and Y calibration point represent the direction of eye 
movements [15]-[16].The eyeball position at the (AO, BO) 
points is: 

AO = (A1+A2) / 2 
BO = (B1+B2) / 2 



VI. RESULTS 

The wheelchair system received the resulted data of image 
processing and based on eye pupil centre location. It sends the 
command to motor driving circuit. Then wheelchair moves in 
required direction according to eye movement. Ultrasonic 
sensor is used in this system for obstacle detection. It measure 
the distance between wheelchair and obstacle. When obstacle 
is very close to wheelchair, motors will stop the wheelchair. 
Central switch is used for emergency purpose to stop the 
wheelchair. Figure 6 shows the Output of the system. 



Figure 6.1: Eye center localization 



Figure 6.2: Eye pupil in left direction 


e) Hough circle transform 
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Figure 6.3: Eye pupil in right direction 



Figure 6.4: Final output of the system 


VII. CONCLUSION AND FUTURE WORK 

A. Conclusion 

In this project we developed a wheelchair system which 
enables the disabled patient to move their wheelchair 
independently in their own direction. In the real time 
application, we can use camera, emergency switch and 
ultrasonic sensor depends on their application. The 
wheelchair movement operation with some delay time. Dark 
light places affect the performance of wheelchair, difficult to 
track the eye pupil in dark light. 

Some of the more unusual aspect of this project is that the 
equipment and parts are all readily available and off the shelf 
including much of the software. The cost for the eye 
controlled wheelchair is slight increase in cost over a normal 
power wheelchair. 

B. Future work 

To make the system more interact with patient we need to add 
some additional sensors. Delay time may be further reducing 
to a second. Operation of system depends on eye movement of 
totally paralyze patients. Thus wheelchair moves in all 
required direction with good response so it will be used in day 
to day life like physically challenged, old people unable to 
walk, physically challenged children. 
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